This uses pointblank to create a data validation report.
In the resulting table at the end, any failing tests should have a CSV
button that lets you download a .csv file of just the rows of data that
don’t pass that particular validation step.
Action levels
For most tests, the following criteria are used:
The exceptions are, —-, —-, —–. These use the strict condition, which will return an ‘error’ if any rows fail.
al_default <- action_levels(warn_at = 1, stop_at = 0.02) #warn if even row fails, error if 2% of rows fail
al_strict <- action_levels(stop_at = 1) #error if even one row fails
HDP_plots.csv and HDP_1997_2009.csv.| Pointblank Validation | |||||||||||||
| Data Validation
tibbleWARN
1
STOP
0.02
NOTIFY
—
|
|||||||||||||
| STEP | COLUMNS | VALUES | TBL | EVAL | UNITS | PASS | FAIL | W | S | N | EXT | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | col_vals_in_set()
|
|
✓ |
67K |
67K1.00 |
00.00 |
— |
○ |
— |
— | |||
| 2 | col_vals_in_set()
|
|
✓ |
67K |
67K1.00 |
00.00 |
— |
○ |
— |
— | |||
| 3 | Height is measured to nearest cm
|
— |
|
✓ |
57K |
57K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 4 | Shoots is interger
|
— |
|
✓ |
57K |
57K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 5 | Number of inflorescences is integer
|
— |
|
✓ |
2K |
2K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 6 | shoots between 0 and 20
|
|
✓ |
67K |
67K0.99 |
80.01 |
● |
○ |
— |
||||
| 7 | height between 0 and 200cm
|
|
✓ |
67K |
67K0.99 |
20.01 |
● |
○ |
— |
||||
| 8 | infloresences between 0 and 3
|
|
✓ |
67K |
67K0.99 |
150.01 |
● |
○ |
— |
||||
| 9 | duplicated rows
|
— | — |
|
✓ |
67K |
67K1.00 |
00.00 |
— |
○ |
— |
— | |
| 10 | col_vals_not_null()
|
— |
|
✓ |
67K |
67K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 11 | Check for duplicate ID's within each year
|
— |
|
✓ |
3K |
3K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 12 | Check for duplicate ID's within each year
|
— |
|
✓ |
4K |
4K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 13 | Check for duplicate ID's within each year
|
— |
|
✓ |
5K |
5K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 14 | Check for duplicate ID's within each year
|
— |
|
✓ |
6K |
6K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 15 | Check for duplicate ID's within each year
|
— |
|
✓ |
6K |
6K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 16 | Check for duplicate ID's within each year
|
— |
|
✓ |
6K |
6K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 17 | Check for duplicate ID's within each year
|
— |
|
✓ |
6K |
6K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 18 | Check for duplicate ID's within each year
|
— |
|
✓ |
6K |
6K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 19 | Check for duplicate ID's within each year
|
— |
|
✓ |
7K |
7K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 20 | Check for duplicate ID's within each year
|
— |
|
✓ |
5K |
5K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 21 | Check for duplicate ID's within each year
|
— |
|
✓ |
6K |
6K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 22 | Check for duplicate ID's within each year
|
— |
|
✓ |
6K |
6K1.00 |
00.00 |
— |
○ |
— |
— | ||
| 2023-05-25 20:49:58 EDT 2.6 s 2023-05-25 20:50:00 EDT | |||||||||||||
Checks that year to year change in size is reasonable
| Pointblank Validation | |||||||||||||
| Check growth & regression
tibbleWARN
1
STOP
0.02
NOTIFY
—
|
|||||||||||||
| STEP | COLUMNS | VALUES | TBL | EVAL | UNITS | PASS | FAIL | W | S | N | EXT | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | |% change in height| < 200%
|
|
✓ |
67K |
66K0.99 |
4200.01 |
● |
○ |
— |
||||
| 2 | |∆ height| < 100cm
|
|
✓ |
67K |
67K0.99 |
110.01 |
— |
● |
— |
||||
| 3 | |∆ shoot number| < 5
|
|
✓ |
67K |
67K0.99 |
2010.01 |
— |
● |
— |
||||
| 2023-05-25 20:50:02 EDT < 1 s 2023-05-25 20:50:02 EDT | |||||||||||||